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HUMAN  PERFORMANCE.  JOBS,  AND  SYSTEMS  PSYCHOLOGY--THE  SYSTEMS  MEASUREMENT  BED* 


BRIEF 


The  major  hypothesis  concerns  the  way  aptitudes,  job  demands,  and  surrounding  conditions 
coalesce  to  yield  varying  levels  of  performance. 

The  conceptual  background  for  the  hypothesis  includes:  1)  a  taxonomy  of  jobs  containing 
cognitive  variance  (responses  more  objectively  characterized  as  right  or  wrong)  and  noncognitive 
variance  (responses  less  objectively  characterized  as  desirable  or  undesirable);  2)  the  ad  hoc 
nature  of  values  and  goals  (quality  output,  safety  of  operation,  quantity  output,  etc.);  3)  the 
great  variety  of  styles  of  behavior  (authoritarian,  forceful,  democratic,  permissive,  cautious, 
risk-taking,  etc.)  by  which  individuals  and  organizations  seek  to  achieve  and  do  achieve  goals. 

Styles  of  behavior  and  values  and  goals  are  considered  ad  hoc  by  the  investigator.  To  the 
extent  that  such  variance  enters  into  criterion  determination,  it  is  proposed  that  for  many  applied 
purposes  including  systems  development,  the  criterion  should  be  a  given  rather  than  the  yield  of 
preceding  predictors,  and  the  criterion  should  be  explicitly  specified  with  respect  to  both  cogni- 
tive  and  noncognitive  variance.  The  systems  measurement  bed-constructed  for  research  purposes 
to  contain  situational  tests  and  exercises-can  be  designed  to  yield  such  criterion  measures  and 
measures  related  to  the  human  factors  variables. 

And  most  importantly,  in  certain  jobs  (probably  more  complex  or  less  structured  jobs)  as 
performed  by  groups  of  individuals  whose  effectiveness  is  greater  than  would  be  expected  based 
on  the  usual  linear  combination  of  empirically  validated  aptitude  measures,  there  occurs  an 
intensification  of  capability  that  goes  beyond  the  use  of  a  relatively  high  level  of  one  aptitude  to 
compensate  for  a  lesser  level  of  another  aptitude.  This  major  hypothesis,  tested  and  supported  in 
appropriate  samples,  highlights  the  complex  of  performance  as  a  functic  i  of  abilities.  Crucial  to 
productive  experimentation  in  the  framework  of  the  systems  measurement  bed  is  the  participation 
of  performers  who  are  experienced  and  even  "expert"  in  the  job  situation  under  scrutiny. 

The  theoretical  discussion,  the  hypothesis  developed  and  tested,  and  the  resultant  research 
considerations  strongly  affirm  that  the  generation  of  principles  for  the  understanding  of  pertinent 
human  behavior  in  certain  jobs  (e.g.,  electronics  specialists,  high  level  jobs  in  complex  informa¬ 
tion  systems,  military  commanders  and  leaders)  seems  to  demand  a  special  research  framework. 
The  research  framework  which  meets  such  a  requirement  is  the  systems  measurement  bed. 


*This  is  an  abstract  of  the  presidential  address  given  by  Dr.  J.  E.  Uhianer  before  the  Division  of 
Military  Psychology  of  the  American  Psychological  Association  at  the  annual  convention  in 
Miami,  Florida  in  September  1970. 
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HUMAN  PERFORMANCE,  JOBS,  AND  SYSTEMS  PSYCHOLOGY-THE  SYSTEMS  MEASUREMENT  BED 


From  the  very  beginning  of  the  establishment  of  U.  S.  military 
psychology  in  I9I7  there  has  been  a  continuing  preoccupation  with  the 
prediction  of  effective  military  behavior,  particularly  with  the  pre¬ 
diction  of  adequate  and  expert  job  performance.  Under  the  leadership 
of  Dr.  Robert  M.  Yerkes,  the  Army  Alpha  and  Beta  were  intended  as  an 
aid--and  indeed  did  aid--in  meeting  the  personnel  management  require¬ 
ments  of  accepting  or  rejecting  and  sorting  personnel.  This  preoccupa¬ 
tion  with  prediction  was  carried  into  the  World  War  II  era  which  saw 
AGCT  developed  under  the  leadership  of  Dr.  Walter  Van  Dyke  Bingham  and 
the  beginnings  of  differential  classification  efforts  when  the  U.  S. 

Army  and  the  U.  S.  Army  Air  Corps  attempted  to  gain  greater  useful 
capability  from  the  total  manpower  pool  (Uhlaner,  1968). 

By  I950,  the  armed  services  had  moved  to  the  development  and  use 
of  a  common  screening  procedure  for  induction  and  enlistment,  the  Armed 
Forces  Qualification  Test  (Uhlaner,  1952). 

Preoccupation  with  the  prediction  of  a  variety  of  human  behaviors 
after  World  War  II  led  to  a  broadened  interest  in  the  management  of 
manpower  through  training  and  planned  job  experience  ( changing  the  per¬ 
former).  With  greater  sophistication  of  machine  and  weapons  technology, 
human  engineering  or  modification  of  the  job  and  surrounding  conditions 
(changing  the  stimulus)  became  a  necessity.  More  recently,  the  con¬ 
cepts  of  organizational  psychology  (changing  the  noncognitive  stimuli) 
have  gained  popularity. 

This  paper  concerns  itself  with  jobs,  human  factors  variables,  and 
systems  psychology  with  the  objective  of  predicting  and  managing  military 
human  performance  in  the  present  era. 

Joining  many  of  my  colleagues  in  industrial  and  military  psychology, 
I  have  been  increasingly  aware  that  for  many  jobs,  and  particularly  those 
involving  noncognitive  elements,  ov.r  prediction  research  has  not  been 
yielding  the  substantial  predictability  so  desired  by  both  management 
and  researchers. 

My  concern  is  with  the  limited  usefulness  of  Information  coming  out 
of  many  personnel  research  studies,  particularly  research  studies  dealing 
with  selection,  the  prediction  of  human  performance,  and  the  measurement 
of  aptitudes  and  abilities  for  differential  classification  (Uhlaner, 
1967)*  This  concern  has  been  voiced  by  others.  For  example,  Ghlselli 
(I966)  has  recently  stated  that  "though  some  few  specific  tests  do  give 
reasonably  good  prediction  of  job  proficiency  in  the  industrial  occupa¬ 
tions  as  a  whole,  the  general  picture  is  one  of  quite  limited  predictive 
power." 
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Various  reasons  are  given  for  the  low  validity  coefficients  obtained: 

1.  The  studies  were  carried  out  on  present  employees  after  they  had 
been  on  the  job  long  enough  so  that  the  gross  failures  had  been  eliminated, 
thus  restricting  the  range  of  ability  both  on  the  tests  and  on  the  job. 

2.  The  criterion  was  highly  unreliable  or  fractional  or  impossible 
to  measure  in  its  appropriate  entirety. 

3.  The  predictive  validation  supplied  answers  which  no  longer 
applied  to  the  current  situation. 

To  my  way  of  thinking,  such  "reasons"  tend  to  perpetuate  the  fond 
hope  that  the  next  study  will  support  the  assumption  that  the  true 
validity  coefficients  are  higher,  repeating  the  usual  predictor-criterion 
covariance  cycle  again  and  again. 

The  researcher  has  a  tendency  to  emphasize  the  psychological  content 
and  approach  with  which  he  has  become  accustomed  to  dealing  and  which 
have  tended  to  yield  "successful"  results  (as  with  the  drunk  who,  having 
dropped  his  keys  in  the  dark  alley,  takes  up  the  search  for  them  under 
the  lamppost)  without  adding  much  in  the  way  of  new  understanding  and 
insight.  Such  research. has  tended  to  yield  fractionated  approaches  which 
have  left  the  decision-maker  less  and  less  satisfied  with  the  human 
factors  systems  impact  (Uhlaner  and  Drucker,  1964). 

If  the  military  research  psychology  specialty  is  selection,  then  the 
promise  has  been  that  the  inventive  and  technical  genius  of  the  researcher 
will  devise  tests,  instruments,  and  other  predictors--cognitive  and  non- 
cognitive--which  will  have  meaningful  validity  for  the  criterion  of 
effective  behavior  and  effective  work  performance. 

If  the  military  research  psychology  specialty  is  training,  then  the 
promise  has  been  that  the  inventive  and  technical  genius  of  the  researcher 
will  devise  training  methods  and  programs,  whether  at  the  school  or  on 
the  job,  for  reasonable  cost,  which  will  offer  the  sponsor  the  means  of 
modifying  the  behavior  of  the  trainee  and  of  providing  the  trainee  with 
appropriate  knowledge  and  skills,  irrespective  of  the  abilities  he  starts 
with. 


If  the  military  research  psychology  specialty  is  human  engineering, 
then  the  promise  has  been  that  the  inventive  and  technical  genius  of  the 
researcher  will  devise  the  appropriate  modification  of  stimulus  or  stimuli 
relevant  to  a  specific  man-machine  system  so  as  to  make  it  possible  for 
the  man  to  perform  effectively--perhaps  irrespective  of  limitations  of 
ability  and  training,  and  ideally  at  minimum  cost. 

If  the  military  research  psychology  specialty  is  organizational 
psychology,  perhaps  with  a  dash  of  social  psychology,  then  the  promise 
is  that  the  inventive  and  technical  genius  of  the  researcher  will  devise 
appropriate  modifications  of  the  "motivational"  environment;  such  modi¬ 
fications  would  utilize  incentive  and  leadership  styles  so  as  to  lead  to 
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more  effective  behavior  and  work  performance  of  teams  and  groups  and 
military  organizations  irrespective  of  the  individual  abilities  of  the 
workers,  their  training,  and  the  engineering  excellence  of  the  machine. 


Fundamental  to  the  understanding  of  the  human  factors  system  is 
the  recognition  that  the  effective  behavior  and  work  performance  desired 
in  a  particular  situation  may  be  achieved  in  a  great  variety  of  ways-- 
admittedly  for  varying  costs  (Schema  I).  If  more  talented  applicants 
are  selected,  the  treatment  (training)  accorded  them  may  be  simplified, 
and  the  equipment  may  require  less  human  engineering  attention.  On  the 
other  hand,  less  capable  individuals  may  require  longer  and  more  skillful 
treatment  (training  and  experience)  and  better  human-engineered  equipment 

Not  only  are  there  interactions  between  selection  variables  an'd 
treatment  variables  (training  and  experience)  and  job  design  variables 
(equipment,  environment,  and  organization),  but  there  are  interactions 
within  a  domain,  such  as  between  cognitive  and  noncognitive  variables 
within  the  selection  domain.  Any  of  the  above  conditions  and  resulting 
performance  may  be  further  modified--one  way  or  another--by  the  kind  of 
supervision  and  leadership  provided  in  the  work  situation  and  by  many 
other  variables  too  numerous  to  mention  here. 

The  amount  of  research  reported  which  examines  these  interactions 
is  thus  far  quite  meager.  A  few  efforts  studying  such  interactions  have 
appeared  and  point,  in  my  opinion,  to  the  fruitfulness  of  examining  more 
extensively  interactions  and  trade-offs  of  human  factors  system  variables 

In  its  1969  SPECTRUM  studies,  HumRRO  examined  the  interactions 
between  aptitude  level  and  training  techniques  for  tasks  of  varying  com¬ 
plexity.  The  results  can  be  viewed  as  pointing  up  some  practical  con¬ 
siderations  on  which  to  base  trade-offs  between  time  and  effort  spent  in 
training  and  level  of  performance  desired.  Men  of  lower  aptitude  re¬ 
quired  more  training  time  to  reach  a  specified  level  of  performance,  as 
well  as  different  instructional  methods--more  guidance,  more*  repetition, 
more  prompting — than  did  the  upper  level  groups.  On  a  composite  measure 
of  Basic  Training  performance,  33  percent  of  the  low  aptitude  groups  com¬ 
pared  to  66  percent  of  the  high  aptitude  groups  scored  above  the  median. 
In  some  tasks,  groups  differed  only  in  rate  of  learning;  in  others,  they 
differed  both  in  rate  of  learning  and  final  level  of  performance.  Not 
all  recruits  labeled  "low  aptitude"  were  slow  learners  on  all  tasks.  On 
each  task,  a  few  showed  performance  typical  of  middle  or  high  aptitude 
groups.  To  quote  the  authors: 

"The  college  graduate  and  the  low  Category  IV  simply  cannot  be 
reached  by  the  same  instructional  vehicle.  However,  these  same  studies 
confirm  the  meagerness  of  current  knowledge  on  how  to  go  about  designing 
appropriate  strategies  for  the  various  levels  of  aptitude.  This  is  a 
serious  gap  in  the  technology  of  training  and  education"  (Fox,  Taylor 
and  Caylor,  I969). 
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In  1964,  from  data  on  48  Army  School  courses,  estimates  were  made 
in  a  BESRL  study  of  the  expected  success  in  training  of  enlisted  men 
with  AFQT  scores  in  the  10th  to  JOth  percentile  (Category  IV).  It  was 
estimated  that  substantial  numbers  of  Category  IV  men  could  successfully 
complete  Army  School  courses  for  MOS  of  moderate  or  low  difficulty  level. 
However,  to  keep  attrition  in  a  given  course  at  a  permissible  level, 
supplementary  aptitudes  in  specified  areas  would  have  to  be  required 
(Helme,  1964). 

Now  I  would  like  to  move  to  another  type  of  job  which  enables  me 
to  highlight  a  set  of  interactions  other  than  the  ones  just  discussed. 

The  job  of  leader  well  illustrates  the  need  to  consider  carefully 
the  influence  of  the  ad  hoc  situation  and  styles  of  behavior  on  job  per¬ 
formance.  By  ad  hoc  situation  is  meant  the  specification  of  the  purposes 
and  conditions  that,  over  a  short  or  a  long  period  of  time,  demand  an 
action.  Values  and  styles  of  behavior  may  dictate  the  action.  As  styles 
of  behavior  may  be  persistent  or  changeable,  so  the  ad  hoc  situation  may 
vary  as  a  function  of  innumerable  factors  including,  most  Importantly, 
value  patterns.  As  in  virtually  all  jobs  where  the  noncognitlve  element 
is  prominent,  the  leader  has  choice  of  actions  in  achieving  the  mission 
objective.  How  he  goes  about  carrying  out  the  objective  is  dictated  in 
part  by  his  style  of  behavior,  in  part  by  his  value  system,  and  in  part 
by  the  ad  hoc  situation. 

The  job  of  leader  has  high  noncognitlve  demands;  thus,  there  exists 
a  great  variety  of  styles  of  behavior  and  value  system  combinations  for 
achieving  goals.  In  addition,  the  variations  in  value  systems  on  the 
one  hand,  and  in  situations  on  the  other  lead  to  numerous  varieties  of 
behavior  that  a  leader  can  adopt  to  achieve  his  objectives.  This  reason¬ 
ing  holds  even  when  the  leaders  have  all  been  brought  to  a  minimum  ade¬ 
quacy  level  of  cognitive  competence  by  initial  selectivity  and  training.1-' 
My  model  does  not  assume  one  style  of  leadership  per  se  as  right  or  wrong 
nor  one  style  of  leadership  aR  required  to  achieve  a  given  goal,  unless 
the  other  factors  of  the  ad  hoc  situation  and  other  pertinent  parameters 
are  specified.  Authoritarian  versus  participative  management  styles, 
for  example  (Schema  II ),  in  themselves  are  not  considered  effective  or 
ineffective  styles. 

It  is  believed  that  if  noncognitive  and  situational  elements  are  to 
be  taken  into  account,  an  experimental  research  environment  must  be 
created  which  lies  somewhere  between  school  and  job.  We  need  an  effec¬ 
tive  way  of  relating  the  noncognitive  with  the  cognitive  in  a  design 
which  makes  it  possible  to  specify  at  the  start  a  great  many  of  the 
interactions  dictated  by  the  applied  setting. 


Actually,  we  knew  that  in  laarnino  a  apecific  body  of  knowledge,  cognitive  meaaurea  ahow  dacraaaad  validity  aa  mote 
and  more  learning  takaa  place.  Whan  a  group  of  individuate  have  all  been  brought  to  a  minimum  edeeuate  level  of 
cognitive  competence,  then  a  boat  of  fectora  other  than  cognitive  lead  to  differencea  in  performance. 
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The  research  approach  proposed  utilizes  the  systems  measurement  bed. 
In  a  sense,  the  military  has  for  a  long  time  intuitively  designed  systems 
measurement  beds.  A  good  example  is  the  military  maneuver,  which  attempts 
to  deal  with  a  large  number  of  factors,  including  men,  equipment,  organi¬ 
zations,  tactics,  and  doctrine  to  test  operational  readiness.  Early  in 
the  1950's,  BESRL  began  an  effort  using  such  concepts  in  the  study  of 
relationships  involving  team  performance  within  squads.  Of  specific 
interest  here  is  that  such  an  approach  helped  in  the  measurement  of  the 
performance  of  the  squad  (Havron,  Fay,  and  McGrath,  1952). 

The  school  setting  gives  highly  useful  results  where  cognitive  con¬ 
tent  is  dominant.  But  for  the  study  of  many  important  jobs,  a  setting 
is  required  in  which  noncognitive  content  can  make  its  influence  felt. 

In  devising  a  systems  measurement  bed,  we  steer  a  course  somewhere 
between  the  school  and  the  real  job  situation. 

Establishment  of  a  systems  measurement  bed  calls  for  a  great  deal 
of  subject-matter  expertise.  Situations  have  to  be  designed,  scenarios 
written,  measurement  strategies  devised,  computer  programs  prepared. 
Appropriate  experienced  personnel  have  to  be  identified  to  serve  as 
subjects.  Finally,  all  these  concepts,  materials,  and  procedures  have 
to  be  built  into  a  logistically  feasible  physical  space,  either  a  com¬ 
puterized  laboratory  or  a  large  terrain  suitably  varied  and  instrumented. 
In  such  a  bed,  we  can  build  in  the  ad  hoc  factors  we  feel  are  most 
relevant  and  especially  the  specifications  of  the  criteria. 

The  systems  measurement  bed  can  bring  together  the  job  and  perform¬ 
ance  variables  of  importance  in  a  research  setting  for  the  purpose  of 
gaining  Insights  for  predicting  and  managing  (military)  behavior  in 
furtherance  of  specified  goals  (missions).  It  is  particularly  vital  to 
represent  noncognitive  as  well  as  cognitive  input  and  to  guard  against 
the  tendency  to  reduce  observation  and  measurement  to  a  preponderance 
of  the  cognitive. 

The  type  of  results  obtained  with  a  systems  measurement  bed  is 
indicated  in  Schema  III.  Factor  analysis  (Helme,  I970)  of  measures  of 
specific  leadership  behaviors  obtained  in  a  situational  context  yields 
a  better  understanding  of  the  military  leadership  job.  The  behaviors 
which  showed  some  generality  across  more  than  one  activity  fell  into  two 
major  domains,  combat  leadership  on  the  one  hand  and  technical /managerial 
on  the  other.  For  both  kinds  of  leader,  there  was  heavy  cognitive  con¬ 
tent,  represented  in  the  combat  leader  by  his  knowledge  of  tactical 
skills,  in  the  technical /managerial  leader  by  technical  skills.  In  the 
combat  leader,  the  heavy  noncognitive  element  rested  upon  forcefulness 
in  the  command  of  men,  personal  resourcefulness,  and  persistence  in 
accomplishment  of  the  mission.  The  noncognitive  element  in  technical/ 
managerial  leadership  was  evident  in  an  Executive  Direction  factor, 
plus--as  in  combat  leadership--a  Mission  Persistence  factor.  The 
specific  behavioral /situational  loadings  on  the  factors  shown  in  Table  1 
are  believed  to  be  very  useful  in  providing  basis  for  the  sunmary  state¬ 
ment  in  the  middle  of  Schema  III,  namely,  that: 
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"High"  leaders  use  appropriate  behavior  style  and  content,  and 
initiate  structure  effectively,  taking  into  account  the  characteristics 
of  the  task,  of  those  led,  and  of  the  situation— applying  the  correct 
amount  and  type  of  consideration. 

Other  examples  of  systems  measurement  beds  as  developed  by  BESRL 
are  in  the  areas  of  surveillance  systems  (Figure  1)  and  command  and  con¬ 
trol  systems.  Earlier  similar  efforts  which  led  to  this  kind  of  approach 
were  the  OSS  of  World  War  II,  the  Air  Force  systems  research  efforts 
carried  out  by  Rand  and  continued  by  the  Systems  Development  Corporation 
(Chapman,  Kennedy,  Newell,  and  Biel,  1959),  and  the  Bell  Telephone 
Company  assessment  centers  (Bray,  1966^ 

To  date,  such  simulated  environments  have  tended  to  tap  cognitive 
elements  more  than  noncognitive.  Emphasis  on  obtaining  objective 
measures-- in  many  ways  our  traditional  heritage,  and  a  powerful  one— has 
led  inevitably  to  emphasizing  problems  that  have  a  high  agreement  right- 
wrong,  hence  favoring  cognitive  problems.  To  build  the  noncognitive 
into  systems  measurement  beds,  taking  into  account  styles  of  behavior 
and  specified  situations  of  the  job  environment-- forcefulness  in  command 
of  men,  risk-taking  in  executive  behavior,  deliberate  cautious  driving — 
requires,  in  my  opinion,  a  major  shift  in  our  thinking.  At  this  point, 

I  believe  we  have  a  tendency  even  in  the  systems  measurement  bed  to  tap 
cognitive  variance— in  the  interest  of  reliable  measurement,  of  course. 

As  Indicated  in  Schema  IV,  we  can  now  state  that  the  systems 
measurement  bed  enables  the  researcher  to  specify  required  styles  of 
behavior  and  goals  to  yield  human  factor  systems  data. 

Quite  fundamental  to  an  understanding  of  human  performance  effective¬ 
ness  in  systems  settings  (Uhlaner,  i960)  is  the  job—or  more  accurately 
the  tasks  which  constitute  the  Job  demands.  As  shown  in  Schema  IV,  a 
major  classification  along  cognitive-noncognitlve  dimensionality  is  pro¬ 
posed.  Although  conceptually  every  job  has  both  cognitive  and  noncogni¬ 
tive  aspects,  these  exist  in  differing  degrees.  '  As  a  working  definition, 
the  cognitive  content  of  a  job  consists  of  right  or  wrong  responses, 
while  the  noncognitive  has  an  ad  hoc  quality  consisting  of  styles  o.f 
behavior  and  value  judgments  often  subjectively  determined.  The  latter 
behavioral  interactions  influence  the  individual  or  organizational  goals 
and  relate  to  individual  or  group  hypothesized  expectancies. 

When  I  speak  of  the  cognitive  content  of  a  job,  I  am  concerned  with 
a  right  or  a  wrong  response  to  a  stimulus.  Once  a  few  rules  are  set, 
most  observers  would  agree  to  the  number  of  X's  on  a  screen,  or  to  the 
correct  sum  of  a  column  of  figures,  or  to  a  particular  solution  to  an 
equation.  Of  course,  the  rightness  or  wrongness  is  not  absolute.  The 
probability  that  the  consensus  is  correct  may  be  set  at  some  agreed 
upon  high  p,  of  say  ,99, 
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Type  of  Characteristics 

Leadership  Factors  of  Abilities  Valid  Measures  of  the 

Required  or  Behaviors  of  Aptitudes  Training  Thru  Schooling  Training  on  the  Job  Leadership  Task 
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Schema  III.  Selected  human  factor  variables  related  to  combat  and  technical  military  leadership 


FACTORS  IN  LEADERSHIP  BEHAVIOR  IN  A  VARIETY  OF  SITUATIONAL  DEMANDS 


From  Helms,  Wm.  H.  Identification  end  career  development  of  officer  leaders.  BESAL  Briefing  Supplement.  July  1970. 

’Factor  loadings  from  factor  analysis  of  measures  of  performance  in  Officer  Evaluation  Center  situational  exercise  under  simulated  combat  conditions. 


T«  joa  T>C  KTIAN  FACTORS  SYSTCT  ATS  TKOOCT  1C  ALLY  EWtCTED  OUTCOftS 


IV.  Cognitive  and  noncognitive  job  variance  reflected  in  the  systems  measurement  bed 


It  Is  my  contention  that  when  we  speak  of  the  noncognitive  content 
of  a  job,  we  should  seriously  depart  from  the  cognitive  concept  of 
rightness  or  wrongness.  In  the  noncognitive  domain,  we  are  confronted 
with  a  host  of  value  judgments--colored  by  emotional  ity — which  in  the 
abstract  are  neither  right  nor  wrong,  correct  nor  incorrect.  Many  of 
these  value  judgments  tend  to  be  bipolar  in  concept.  For  example,  a 
person  may  use  power  or  conciliation  in  attempting  to  achieve  certain 
ends.  Quantity  may  be  viewed  as  preferable  to  quality  or  quality  pre¬ 
ferable  to  quantity.  Daring  and  risk-taking  may  be  preferred  over  care 
and  caution,  or  vice  versa.  For  many  of  these  styles  of  behavior,  there 
is  usually  a  good  or  a  bad  connotation  situationally  which  can  easily 
flip-flop  with  another  step  up  the  scale.  Daring  and  risk-taking  may 
move  to  recklessness;  caution  may  move  to  timidity.  Moreover,  it  is  felt 
that  it  is  more  useful  to  replace  the  general  concepts  of  caution,  force¬ 
fulness,  and  other  such  styles  of  behavior  or  traits  with  greater  speci¬ 
ficity,  such  as:  forceful  coimand  of  men  in  combat  or  cautious  approaches 
to  design  of  experiments. 

Schema  IV  shows  a  model  relating  such  concepts  to  jobs,  human 
factors  systems  variables,  and  a  systems  measurement  bed  based  on  such 
conceptualisation . 

The  relative  signiflcance--at  various  stages  of  training--of  the 
cognitive  and  noncognitive  content  of  a  job  and  the  resulting  interaction 
accounts  for  major  differences  in  jobs  with  respect  to  predictability. 

In  a  highly  structured  job  such  as  key  punch  operator,  the  tasks  in¬ 
volved  require  fairly  simple  but  definite  patterns  of  response  to  stimuli. 
Although  some  noncognitive ' factors  may  enter  into  the  level  of  perform¬ 
ance^  of  the  Job,  there  is  fairly  definite  agreement  as  to  right  and 
wrong  responses.  The  criterion  for  such  Jobs  is  easily  specified  and 
performance  is  readily  predictable. 

A  second  major  concept  in  the  systems  psychology  approach  to  the 
measurement  of  human  performance  in  job  settings  is  specification  of  the 
criterion  variance  which  is  to  be  predicted.  The  concern  here  is  not 
with  the  type  of  criterion  measures  Buch  as  ratings,  school  grades, 
product  measures,  or  the  like  (Uhlaner  and  Drucker,  I964).  The  concern 
is  with  value  differences.  Should  quality  be  produced,  perhaps  at  the 


In  any  job,  howavar  almpla,  a  ganaral  factor  of  good  work  adjuatmant  or  poor  work  adjustment  oparataa.  Good  work 
adiuatmant  may  be  ralatad  to  a  ganaral  good  adjuatmant  factor  tranalatad  ganaraily  Into  "playing  tha  gama  while 
obaervlng  tha  rulaa  raaaonably  wall."  Ona  can  think  of  euch  an  adjuatmant  factor  aa  a  ganaral  ovarride  for  almoet 
any  work,  aa  la  alao  tha  variability  aaaociatad  with  organizational  variable*-- laadarahip,  incantivaa,  and  ttylae  of 
managament.  In  moat  joba  of  thla  laval  of  complexity,  what  la  likaly  to  happan  ia  that  an  individual  with  a  cartain 
aat  of  attitudea,  Intaraata,  or  valuaa  will  ba  aalaetad  and  will  or  will  not  accapt  tha  job,  aay,  of  kay  punch  oparator. 
Sut  once  ha  la  in  tha  job.  It  ia  hia  cognitive  capability  which  will  tend  moat  of  all  to  detar  mine  whether  or  not  hia 
performance  I*  adaquata-or  that  tha  job  ia  being  correctly  performed. 
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expense  of  quantity  or  vice  versa?  Should  life  be  carefully  protected 
and  safeguarded  at  considerable  cost?  Should  the  future  be  emphasized 
or  should  the  resources  be  allocated  to  the  here  and  now?  It  is  evident 
that  different  costs  may  be  involved  in  each  of  the  different  criterion 
choices.  The  value  judgment  by  the  decision  maker  as  to  what  the  crite¬ 
rion  variance  make-up  should  be  determines  much  of  the  rest  of  the  human 
factors  system.  Who  makes  this  decision?  Who  decides  to  build  quality 
and  sacrifice  quantity  or  vice  versa?  The  temptation  is  to  say,  the 
owner  of  the  factory,  the  executives,  the  commander.  But  how  much  does 
the  individual  producer  set  his  own  standards,  style,  and  quality  con¬ 
trols?  Upon  a  few  moments  of  reflection,  it  becomes  clear  that  it  is  a 
much  more  complex  process,  varying  with  the  total  cultural-political- 
economic  value  complex  of  a  nation.  But  my  thrust  here  is  that,  for 
many  applied  purposes  including  systems  development,  the  criterion  should 
be  a  given  rather  than  the  product  of  a  preceding  set  of  given  predictors. 

In  image  interpretation,  for  example,  the  information  output--which 
can  be  considered  the  criterion--can  emphasize  either  accuracy  (percent 
correct  information  reported  by  an  image  interpreter)  or  completeness 
(percent  correct  information  extracted  from  an  image  in  terms  of  "what 
is  extractable").  Based  on  empirical  data,  investigators  have  shown 
that,  while  there  is  an  increase  in  completeness  as  a  function  of  in¬ 
crease  in  time,  there  is  a  decrease  in  cumulative  accuracy  (Figure  2). 

In  the  systems  measurement  bed,  under  one  set  of  requirements,  the  crite¬ 
rion  may  be  set  to  emphasize  accuracy,  as  in  the  case  of  a  commander  who 
has  to  pinpoint  critical  targets  interspersed  with  noncritical  targets. 
Under  another  set  of  requirements,  the  emphasis  may  be  on  completeness, 
irrespective  of  criticality.  The  human  factors  system  variables  leading 
to  these  different  criterion  outputs  can  then  be  examined  in  relation  to 
the  type  of  men  required,  the  procedures  and  work  methods  required  for 
each  of  the  specified  criterion  conditions.  It  is  recognized  that  the 
applied  requirement  is  often  a  mix. 

If  we  keep  in  mind  that  cognitive  elements  of  performance  can  be 
characterized  as  right  or  wrong,  then  schools  are  indeed  effective  envi¬ 
ronments  in  tapping  these  elements,  and  it  is  appropriate  to  validate 
the  cognitive  predictors  for  "noncont rovers ial"  tasks  in  our  school 
training  programs..  We  must,  however,  accept  for  this  purpose  the  "school 
solutions"  in  the  jobs  being  taught.  Further,  prediction  of  Job  perform¬ 
ance  where  the  cognitive  element  is  crucial  has  been  relatively  more 
substantial  because  universality  of  agreement  is  more  readily  attainable. 
The  latter  works  especially  well  when  the  noncognitive  environment  is 
relatively  stable.  Prediction  of  Job  performance  where  the  noncognitive 
element  is  crucial  involves  value  judgments  and  situational  considera¬ 
tions,  yielding  less  agreement  on  desired  outcomes  and  hence  much  smaller 
predictability. 
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Figure  2.  Interpreter  performance  (accuracy  and  completeness)  as  a  function  of  time.  (From  Sadacca.R., 
H.  Martinek,  and  A.  /.  Schwartz.  Image  Interpretation  Teak-Status  Report,  30  June  1962. 

BESRL  Technical  Research  Report  1129.  October  1962 J. 


The  high  prediction  of  achool  auccess  obtained  with  cognitive  pre- 
dlctors  la  easily  understood  In  view  of  the  greater  ease  with  which  right- 
wrong  concepts--in  contrast  with  attltudlnal  and  value  judgments--are 
structured  In  school  curricula.  In  general,  the  cognitive  abilities  are 
well  measured  and  yield  high  validity  for  school  grades — coefficients  In 
the  neighborhood  of  .5  to  .7.  Even  for  on-the-job  criteria,  the  cognitive 
variance  Is  more  easily  tapped.  Research  studies  show  that  raters  can 
assess  with  considerable  agreement  the  cognitive  elements  of  performance. 

An  analysis  of  over  100  studies  of  criteria  and  predictors  reported 
in  Dorcus  and  Jones'  Handbook  of  Employee  Selection,  plus  a  collection  of 
more  recent  validation  studies,  led  Gaylord,  Severin,  and  Johnson  (1952) 
to  the  general  conclusion  that  cognitive  tests  predict  job  performance-- 
whether  evaluated  by  ratings  or  by  production  records--better  than  do  non- 
cognitive  measures  (median  correlation  of  .53  versus  .15).  Thus,  even 
when  both  predictor  and  criterion  emphasize  the  noncognitive,  much  of  the 
cognitive  enters  into  the  relationship. 

Peer  ratings,  and  indeed  ratings  of  most  kinds  including  school 
grades,  yield  a  heterogeneous  variance.  They  are  complex  measures  and 
quite  meaningless  when  independent  of  the  job  situation  or  specific  envi¬ 
ronment.  In  a  forced  environment,  in  which  specific  values  and  styles  of 
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behavior  are  taught  as  preferred  and  in  which  these  values  are  continually 
reinforced,  the  rater  makes  his  judgments  in  relation  to  the  subtleties 
of  the  job  demands  he  has  been  taught.  He  has  absorbed  the  total  systems 
setting  and  what  is  or  is  not  considered  desirable  in  that  setting.  The 
USMA  is  a  good  example  of  such  a  forced  environment.  The  cadet  learns 
what  is  considered  a  desirable  stance  and  gives  this  emphasis  in  his 
rating;  he  does  not,  1  believe,  rate  a  trait  in  the  abstract.  He  judges 
in  terms  of  qualities  the  job  demands  as  he  was  taught  and  how  well  the 
rated  individual,  in  his  judgment,  adjusts  to  these  demands,  and  whether 
the  ratee  utilizes  "desirable"  styles  of  behavior  in  given  settings. 
Predicted  validity  is  generally  high  when  the  rater  has  known  the  ratee 
in  such  Job  situations.  The  longer  the  period  of  association,  the  more 
able  the  rater  is  to  make  such  judgments.  For  example,  performance  of 
officer  trainees  was  better  predicted  for  the  West  Point-trained  officers 
in  combat  in  Korea  than  for  OCS-tralned  officers,  a  difference  attribut¬ 
able  to  the  factors  discussed  above  (Table  2). 

The  crux  of  why  we  get  high  or  relatively  high  covariance  with  the 
aptitude  for  the  service  rating  (a  combination  of  tactical  officer  and 
peer  ratings)  (Drucker,  1957)  Is  that  in  a  forced  environment  such  as 
the  service  academies  the  other  cadets  have  learned  the  desired  noncogni- 
tive  elements  such  as  forcefulness  in  the  specific  settings.  "Obnoxious" 
forcefulness  is  rated  down.  Just  the  right  amount  of  forcefulness  appro¬ 
priate  to  the  situation  tends  to  be  rated  high.  In  other  words,  ratings 
in  a  forced  environment  get  at  the  essence  of  job  demands  specified  in 
that  environment. 

How  does  all  this  differ  from  traditional  research  approaches?  It 
seems  to  me  that  what  we  have  been  doing  in  traditional  personnel  research 
is  to  abstract  certain  traits,  bits  of  cognitive  and  noncognitive  behavior, 
and  relate  them,  singly  or  in  linear  combination,  to  part  of  a  job  crite¬ 
rion.  We  relate  measures  of  reasoning  ability,  sunmatlons  of  self¬ 
descriptors  said  to  describe  the  trait  ascendency,  to  criterion  measures 
of  opportunity,  whether  it  be  performance  in  a  job  specialty  or  a  Job 
generality,  e.g.,  leadership.  In  a  way,  we  have  been  like  the  three 
blind  men  touching  different  parts  of  the  elephant  and  describing  it 
variously  as  a  wall,  a  snake,  and  a  piece  of  rope.  To  have  a  better 
understanding  of  job  performance,  we  have  to  have  the  more  complete  ad 
hoc  aspects  of  the  noncognitive  demands  along  with  the  cognitive  demands 
in  the  appropriate  combination. 

The  appropriate  combination  becomes  crucial  if  it  can  be  shown  that 
there  is  a  residue  of  criterion  variance  which  is  significantly  differ¬ 
ent  from  the  prediction  arising  from  the  linear  combination  of  valid 
predictors. 

Let  us  focus  on  the  simple  problem  of  predicting  end-of-training 
success  with  empirically  determined  predictors.  If  we  restrict  ourselves 
to  the  case  where  we  examine  a  pair  of  predictors,  some  of  the  principles 
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are  more  readily  apparent.  Further,  one  predictor  of  the  pair  is  drawn 
from  the  broader  mental  ability  domain--mathematlcal  apt i tude--and  the 
other  from  the  more  specific  aptitudes--mechanical  aptitude.  Further, 
we  contrast  the  phenomenon  between  more  complex  jobs  (electronics)  and 
less  complex  jobs  (electrical). 

I  should  now  like  to  introduce  the  principle  of  substitutability 
with  intensification  or  dilution  (nonlinear),  adding  it  to  the  existing 
principle  of  substitutability  with  compensation  (linear)  (Guilford,  1954). 

The  principle  of  substitutability  operates  in  several  ways.  In  one 
case,  a  relatively  high  l^vel  of  ability  in  one  area  makes  up  for  a  low 
level  in  another  so  that  observed  performance  equals  that  predicted  from 
a  linear  combination  of  two  predictor  measures.  This  phenomenon  has  been 
defined  as  substitutability  with  compensation.  The  data  base  gives  some 
indications  that  a  further  interaction  is  operating  under  certain  cir¬ 
cumstances  . 

From  an  Army  sample  of  over  20,000  cases,  studied  with  respect  to  a 
variety  of  predictors  in  approximately  100  Jobs  (MOS)  providing  criterion 
measures  in  an  end-of-school  setting,  two  groups  of  samples  were  identi¬ 
fied:  the  first  in  a  variety  of  electrical  jobs  and  the  second  in  a 
variety  of  electronics  jobs.  The  demands  of  the  electrical  jobs  were 
less  complex  than  those  of  the  electronics  jobs.  A  concept  of  substitut¬ 
ability  between  two  predictors,  mechanical  aptitude  and  mathematical 
aptitude,  was  hypothesized.  If  the  observed  criterion  score  is  greater 
than  the  score  predicted  from  the  linear  model  (with  appropriate  beta 
weights,  etc.),  then  the  difference  can  be  considered  substitutability 
with  intensification.  In  other  words,  the  observed  measures  of  perform¬ 
ance  are  higher  than  that  predicted  by  a  linear  combination  of  mechanical 
aptitude  and  mathematical  aptitude.  If  the  observed  criterion  score  is 
smaller  than  the  score  predicted  from  the  linear  model  (with  appropriate 
beta  weights,  etc.),  then  the  difference  can  be  considered  substituta¬ 
bility  with  dilution.  If  predicted  and  observed  performance  means  are 
equal,  then  substitutability  can  be  considered  compensation  in  the  linear 
sense:  that  is,  a  certain  amount  of  mathematics  aptitude  is  substituting 
in  linear  fashion  for  lesser  mechanical  aptitude.  Further,  it  was  rea¬ 
soned  that  Intensification  is  more  likely  to  occur  in  the  more  complex 
jobs.  Table  5  substantiates  the  hypothesis  that  for  more  complex  jobs 
(electronics,  in  this  case),  substitutability  with  intensification  does 
operate.  For  this  analysis,  the  criterion  performance  of  men  relatively 
high  in  mathematics  (minimum  standard  score  110)  but  appreciably  lower 
In  mechanical  aptitude  (at  least  20  points--about  one  standard  deviation) 
was  examined.  If,  for  example,  a  man  had  a  score  of  150  in  mathematics 
aptitude,  his  mechanical  aptitude  score  had  to  be  no  higher  than  110. 

In  this  case,  the  linear  model  fails  to  account  for  the  higher  criterion 
scores  of  this  select  group.  In  less  complex  jobs  (electrical),  with  a 
similarly  select  sample,  the  linear  model  explains  the  variance.  Hence, 
only  compensation  seems  to  operate  here. 
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Table  3 


DIFFERENCES  BETWEEN  OBSERVED  AND  PREDICTED  CRITERION 
SCORES  WHEN  THE  MATHEMATICS  PREDICTOR  SCORE  IS 
SUBSTANTIALLY  (AT  LEAST  ONE  S.D.)  GREATER  OR 
SMALLER  THAN  THE  MECHANICS  PREDICTOR  SCORE 
(Large  Differences  Reflect  Non-Linearity) 
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No  data  are  offered  in  this  paper  for  the  substitutability  with 
dilution  model.  It  seems  likely,  however,  that  the  combination  of 
different  domain  variables—cognitive,  noncognitive—may  yield  even 
greater  substitutability/intensification  and  substitutability/dilution. 

To  the  degree  the  linear  model  operates  effectively,  prediction 
from  one  time  frame  to  another--from  school  to  job,  for  example--is 
appropriate  using  present  variables  and  regression  techniques.  To  the 
degree  the  linear  model  does  not  sufficiently  explain  what  is  going  on, 
the  systems  measurement  bed  gives  one  the  opportunity  to  embed  appro¬ 
priate  subjects  (often  after  extensive  training)  for  appropriate  data 
points . 

In  suninary,  if  we  accept  that  cognitive  elements  of  tasks  and  jobs 
have  a  reasonably  objective  right  or  wrong  specification  but  that  non- 
cognitlve  elements  deal  with  styles  of  behavior  and  values  not  readily 
identified  as  right  or  wrong  except  on  an  ad  hoc  basis,  then  criterion 
measures  containing  such  noncognitive  components  may  have  to  be  speci¬ 
fied  arbitrarily  by  the  policy  maker,  the  decision  maker,  the  "man  with 
authority"  and/or  the  scientist  in  order  to  reach  scientific  conclusions 
as  to  what  is  really  taking  place  under  a  specified  variety  of  criteria 
or  specified  outcomes.  It  is  recognized  that  the  specified  desired 
outcomes  can  have  strong  plus  or  minus  preference  loadings. 

In  the  case  of  complex  jobs  (electronics),  the  combination  of 
mathematics  aptitude  of  higher  level  with  mechanical  aptitude  of  lesser 
level  has  been  shown  to  be  nonlinear  and  in  the  direction  of  intensifi¬ 
cation.  Individuals  who  possess  this  combination  of  predictors,  then, 
will  better  represent  the  group  of  people  for  whom  such  criterion  per¬ 
formance  is  characteristic  than  will  individuals  identified  by  scores 
linearly  derived  (with  beta  weights)  from  a  large  random  sample  of 
individuals  possessing  varying  amounts  of  such  predictors. 

The  research  framework  suggested  is  the  system  measurement  bed. 

The  f ocus  of  inquiry  concerns  itself  with  the  human  factors  system 
variables  shown  in  Schemata  I  through  IV.  These  deal  with  designing, 
selecting,  and  training  for  effective  behavior. 

Once  the  framework  and  the  focus  of  inquiry  are  decided  upon,  we 
must  build  into  the  systems  measurement  bed  job  elements  in  an  inter¬ 
acting  context  to  enable  us  to  study  operations  and  outcomes  so  as  to 
gain  insights  and  data  for  predicting  and  managing  desired  behavior- 
in  our  case,  effective  military  behavior. 

For  such  studies,  it  becomes  crucial,  in  the  opinion  of  this  writer, 
that  the  performers  in  systems  measurement  beds  be  experienced  and  even 
"expert"  performers  in  whom  the  nonlinear  combinations  of  abilities  have 
already  taken  place  by  virtue  of  a  host  of  factors  including  training, 
experience,  and  systems  variance.  By  examining  the  behavior  of  such 
appropriately  trained  and  experienced  performers  in  such  scientifically 
unique  settings,  the  human  factors  system  variables  can  be  more  meaning¬ 
fully  interrelated. 


-  20  - 


Such  studies  involve  the  performers  as  dependent  variables  in  an 
experimental  design  conducted  in  a  specified  situation.  The  results  of 
such  experimentation  are  measured  in  terms  of  the  amount  of  variance 
predicted  in  the  systems  measurement  bed,  variance  which  may  be  accounted 
for  to  only  a  slight  extent,  and  perhaps  not  at  all,  by  the  individual 
differences  in  the  performers.  The  objective:  to  gain  a  better  under¬ 
standing  of  human  performance  in  a  specified  realistic  setting.  The 
systems  measurement  bed  is  a  means  of  focusing  step  by  step  on  the  human 
performance  aspects  of  the  system  to  be  enhanced  and  identifying  the 
interrelationships  of  the  human  factor  system  variables  in  order  to 
determine,  for  a  given  cost,  productivity  under  varying  conditions. 

What  is  proposed  is  a  vehicle  for  looking  at  what  really  happens 
on  a  job  in  a  more  systematic  way.  We  must  move  from  preoccupation  with 
the  covariance  of  abstract  measures--whlch  often  yields  little  in  the  way 
of  scientific  knowledge — to  a  means  of  bringing  into  the  laboratory  a 
better  and  more  meaningful  representation  of  the  jobs  and  systems  to  be 
studied.  Our  psychometric  heritage  of  measurement  techniques  has  pro¬ 
vided  us  with  such  concepts  as  construct  validity  and  predictive  valida¬ 
tion.  The  last  50  years  have  given  us  a  rich  methodology  of  empirical 
methods  and  more  recently  a  sophisticated  base  of  computer  techniques 
which  together  make  simulation  a  more  mature  technology.  With  the 
systems  measurement  bed  and  system  psychology  principles,  we  can  now 
vary  men,  training  and  experience,  management  styles  in  relation  to 
specified  outcomes  and  thus  apply  the  more  rigorous  methodology  of 
experimental  psychology  to  the  study  of  jobs  and  human  performance  in 
systems  settings. 
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relatively  high  level  of  one  aptitude  to  compensate  for  a  lesser  level 
of  another  aptitude.  This  major  hypothesis,  tested  and  supported  in 
appropriate  samples,  highlights  the  complex  of  performance  as  a  function 
of  abilities.  Crucial  to  productive  experimentation  in  the  framework  of 
the  systems  measurement  bed  is  the  participation  of  performers  who  are 
experienced  and  even  "expert"  in  the  job  situation  under  scrutiny. 

The  theoretical  discussion,  the  hypothesis  developed  and  tested, 
and  the  resultant  research  considerations  strongly  affirm  that  the 
generation  of  principles  for  the  understanding  of  pertinent  human 
behavior  in  certain  jobs  (e.g.,  electronics  specialists,  high  level  jobs 
in  complex  information  systems,  military  commanders  and  leaders)  seems 
to  demand  a  special  research  framework.  The  research  framework  which 
meets  such  a  requirement  is  the  systems  measurement  bed. 

This  is  an  abstract  of  the  presidential  address  given  by  Dr.  J.  E. 
Uhlaner  before  the  Division  of  Military  Psychology  of  the  American 
Psychological  Association  at  the  annual  convention  in  Miami,  Florida  in 
September  1970. 
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The  major  hypothesis  concerns  the  way  aptitudes,  job  demands,  and  surrounding  con¬ 
ditions  coalesce  to  yield  varying  levels  of  performance. 

The  conceptual  background  for  the  hypothesis  includes:  l)  a  taxonomy  of  jobs  con¬ 
taining  cognitive  variance  (responses  more  objectively  characterized  as  right  or  wrong) 
and  noncognitive  variance  (responses  less  objectively  characterized  as  desirable  or  un¬ 
desirable);  2)  the  ad  hoc  nature  of  values  and  goals  (quality  output,  safety  of  opera¬ 
tion,  quantity  output,  etc.);  3)  the  great  variety  of  styles  of  behavior  (authoritarian 
forceful,  democratic,  permissive,  cautious,  risk-taking,  etc.)  by  which  individuals  and 
organizations  seek  to  achieve  and  do  achieve  goals. 

Styles  of  behavior  and  values  and  goals  are  considered  ad  hoc  by  the  investigator. 
To  the  extent  that  such  variance  enters  Into  criterion  determination,  it  is  proposed 
that  for  many  applied  purposes  including  systems  development,  the  criterion  should  be  a 
given  rather  than  the  yield  of  preceding  predictors,  and  the  criterion  should  be  ex¬ 
plicitly  specified  with  respect  to  both  cognitive  and  noncognitive  variance.  The  sys¬ 
tems  measurement  bed--constructed  for  research  purposes  to  contain  situational  tests 
and  exerclse8--can  be  designed  to  yield  such  criterion  measures  and  measures  related  to 
the  human  factors  variables. 

And  most  importantly,  in  certain  jobs  (probably  more  complex  or  less  structured 
jobs)  as  performed  by  groups  of  individuals  whose  effectiveness  is  greater  than  would 
be  expected  based  on  the  usual  linear  combination  of  empirically  validated  aptitude 
measures,  there  occurs  an  intensification  of  capability  that  goes  beyond  the  use  of  a 
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